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BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] This invention relates to the arts of imaging and image processing, including 
sensor design and moire reduction technologies. 

5 

Backgroxmd of the Invention 

[0002] Well-known imaging technologies include film (e.g. analog), digital, and 
analog-digital hybrid approaches. Film imaging processes use a set of lenses to focus 

10 an image onto a fihn sheet which is impregnated with grains of material reactive to the 
spectrum to be recorded, such as visible light, infiared ("IR"). or X-ray. The grains 
are randomly arranged in each sheet of fihn, and thus reproduction of the image on the 
developed fihn has a certain resolution based on the size and density of these grains. 
[0003] In digital imaging, a sensor of uniformly-arranged sensing elements is used to 

15 capture "bits" or pixels of the image. Turning to Figure 1, the system components 

(10) of a typical digital camera are shown. In this example, a scene or original item (1) 
is digitally imaged using a two-dimensional array (7) of sensors such as an array of 
charge-coupled devices ("CCD"). The image of the scene is focused onto the array 
(7) by a lens (6), and a shutter (not shown) may be used to provided a specific 

20 duration of exposure. The sensor elements are arranged with uniform spacing into 
rows and columns. 
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[0004] Turning to Figure 2, more details of a typical two-dimensional sensor array 
(7) are shown. The sensor element colunms are uniformly spaced at distance di from 
each other according to a linear function such as: 

5 X-axis position of sensor in column n = Pn== (n-1) • di 

where the array is comprised of N sensor columns, n is the column number 
ranging from 1 through N, and di is the uniform distance between the sensor columns. 
[0005] Likewise, the sensor element rows are uniformly spaced at distance dz from 
10 each other according to a linear ftmction such as: 

y-axis position of sensor in column m = Pm= (m-l) • d2 

where the array is comprised of M sensor rows, m is the row number ranging 
1 5 from 1 through M, and di is the uniform distance between the sensor rows. 

[0006] In many two-dimensional sensor arrays, the row-to-row spacing di may be 
equal to the colunm-to-column spacing di. The number of columns N may be equal 
to the nxmiber of columns M, as well. 

[0007] The sensors may in practice be reactive to any range of electromagnetic 
20 ("EM") spectrum according to the desired application, such as charge-coupled devices 
("CCD") for visible or IR imaging. 
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[0008] Typically, the voltage level on each sensor element is measured and 
converted (e.g. sampled) to a digital value using an analog-to-digital converter. The 
sample value is relative to the amount of electromagnetic energy incident on the sensor 
element. Conversion to digital values are typically performed using an 
5 analog-to-digital converter having sufficient resolution (e.g. data width) for the 
intended application. The digital data set (66) of samples represents a digitized or 
pixelated copy of the image. 

[0009] Additionally, mechanical and/or chemical filtering and band separation of the 
EM spectrum may be performed to produce "separated" data sets, such as use of a 
% 10 color wheel in front of tiie sensor array, or placement of color filters over the sensor 

i 5 

K elements themselves, 

[0010] Moire patterns are artifacts of certain imaging processes which are 

1 1 perceptible to the human eye, but do not represent actual features or details in the 

J j original item imaged. They often resemble Crosshatch halftones across all or a portion 

^ 1 5 of a digital image. 

[0011] For imaging processes in which the original is an analog image, for example a 
photographic subject, moire patterns may appear when certain features align with the 
sensors in the sensor array. For example, a digital photograph of a bug screen on a 
window often produces noticeable moire patterns due to the bug screen's uniform 
20 grid-like features. The resulting apparent pattern is actually an interference pattern 
between the physical spacing of features of the original image and the spacing of the 
sensor array. 
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[0012] Just as in the phenomena of interference patterns between other types of 
signals, visual moire patterns may become apparent at "harmonics" or integral 
multiples of spacing distances of the original image features and the spacing distances 
of the sensor array. For example, if the repeating features of a photographic subject 
5 are focussed onto two-dimensional sensor array having a sensor spacing of 600 dots 
per inch ("DPI") and a moire pattern forms, then the same image focused at the same 
distance on a sensor array having a sensor spacing of 1200 DPI will Ukely result in the 
appearance of moire patterns. Undersampling the image at 300 DPI would also likely 
y result in the appearance of moire patterns. 

10 [0013] Many techniques have been developed to try to reduce moire patterns which 

■^jsy ii 

'% appear in existing digital images, such as appUcation of image processing techniques 

I'' including Gaussian blurring, "descreening" algorithms, and "de-speckle" processes, 

ly Most of these have a result of reducing the sharpness of the overall image because 

y they reduce the moire pattern by spreading energy or brightness from a given pixel to 

Q 

1 5 adjacent pixels. 

[0014] For example, ttmiing to Figure 3a, pixel N represents a pixel of a moire 
pattern in a single row or column, and in this case, a pattern which is darker than the 
surrounding pixels, N-1 and N+1. The energy E2 of pkel N is lower than the energy 
E3 of the adjacent pixels N+1 and N-1. A blurring process applies a partial or 

20 weighted averaging among regional or adjacent pixels, such as shown in Figure 3b, 
wherein the energy of the pixel in the moire pattem is slightly increased to E2', and the 
energy of the adjacent pixels are slightly decreased to £3'. 
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[0015] While this oftentimes decreases the obviousness or appearance of the moire 
pattern to the human observer, it also reduces the "sharpness" or level of apparent 
detail of the entire image. If the blurring process is applied manually on a regional 
basis, the degradation to the entire image may be avoided, but the local areas are still 
5 degraded and substantial human intervention may be required to do so. Additionally, 
"edge effects" may become perceptible where the region of processing meets a region 
of unprocessed image. 

[0016] So, to date, most digital image post-processing attempts to reduce moire 
patterns either result in image degradation, require substantial human operator effort, 

10 or both to some degree. 

[0017] A common technique employed to avoid the generation of moire patterns in 
the imaging process is to dither the sensor array such that the array is moved in 
physical position with respect to the original subject being imaged. In Figure 4, such a 
dithering imaging system with a two-dimensional sensor array (7) is shown. An x-axis 

1 5 mechanical jitter drive (40) is coupled to the array (7) such that it's x-axis position is 
varied slightly over time, usually in a sinusoidal or triangular pattern (41). Likewise, a 
y-axis jitter drive (42) may jitter the array ia an orthogonal direction, also typically in a 
sinusoidal or triangular patter (43). 

[0018] This jittering action allows the array (7) to scan a pattern of points which are 
20 not simply an array of uniformly spaced rows and columns, but which represent 

positions relative to the dithering functions Px'(t) and Py'(t). As such, fewer original 
image sources will have an interference pattern with the dithered sensor pattern, but it 



-7- 



AUS920011012US1 Patent AppUcation 

is still possible that portions of the original image source may interfere with the 
dithered sensor pattern to cause localized moire patterns. Additionally, such dithering 
mechanisms tend to add expense and failure rate to an assembly such as a digital 
camera. 

[0019] Therefore, there is a need in the art for a system and method which avoids 
generation of moire patterns in digital images created with a two-dimensional sensor 
array, without the use of mechanical dithering mechanisms, intensive image 
post-processing technologies, or a high degree of human operator manipulation and 
editing. Further, there is a need in the art for this new system and method to maintain 
image quality, while being readily realizable using current sensor technology, and to 
preferrably be compatible with widely-used image compression and decompression 
technologies such as bitmap, JPEG (joint photographic experts) and MPEG image 
products. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[0020] The following detailed description when taken in conjunction with the figures 
presented herein provide a complete disclosure of the invention. 
[0021] Figure 1 illustrates the arrangement of components of an imaging system 
which employs a two-dimensional sensor array, such as a digital camera. 
[0022] Figure 2 shows details of a two-dimensional sensor array. 
[0023] Figures 3a and 3b provide illustration of blurring processes often used to 
reduce the appearance of moire patterns in digital images. 

[0024] Figure 4 shows the mechanical dithering action on a typical two-dimensional 
sensor array. 

[0025] Figure 5 illustrates the sensor placement for a two-dimensional sensor array 
according to the invention. 

[0026] Figure 6 sets forth the logical process of the invention to yield a data set 
representing uniformly-spaced sample points from the non-uniformly spaced sensor set 
according to the present invention. 

[0027] Figure 7 graphically depicts the interpolation process of the preferred 
embodiment. 
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SUMMARY OF THE INVENTION 
[0028] The present invention provides a means for avoiding moire patterns in 
digitized images by employing a two-dimensional sensor array of non-uniformly 
spaced sensors. This allows the spacing of the sensors to avoid having an inherent 
"jfrequency" that may interfere with details or harmonics present in the image source, 
which eliminates the occurrence of moire patterns and the need for application of 
image processing to remove moire patterns. 

[0029] The sensors are placed along each axis in a non-uniform manner according to 
a predetermined scheme or function. During imaguig, sensors are sampled and stored 
into a data set which represents non-uniformly spaced image points within the original 
image, scene or subject Finally, linear interpolation is preferrably applied to the 
non-uniformly spaced data set to yield a synthesized uniformly-spaced data set for use 
ia common imaging formats and processing, such as JPEG or MPEG compression and 
decompression. 
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DETAILED DESCRIPTION OF THE INVENTION 
[0030] The present invention is realized first by fabricating a two-dimensional array 
of sensor elements wherein tiie rows and columns of die array have non-uniform 
spacmg accordmg to a predetermined function. The sensors may be any useful type of 
sensor appropriate for the desired appUcation, such as visible light or IR sensors 
(CCD), or other types of sensors. 

[00311 Turning to Figure 5, the arrangement of sensors in a two-dimensional array 
according to the present invention is shown. The spacing between adjacent sensors in 
the X and y axes are non-uniform, which avoids the physical creation of a "frequency" 
in the sampling operation. 

[0032] For example, where c is the column number of a sensor in a given row r, 
d,=i is not equal to dc=2, and d.^ is not equal to dc=3, and so forth. Further, tiiese 
distances should not be harmonic multiples of each other, such as having dc=i = (1/2) 
dc=2 and dcHj = (1/4) de=3, etc. As such, tiie function used to determine tiie 
sensor-to-sensor spacing of the sensors within a row (e.g. tiie column-to-column 
spacing in the x-axis) of tiie two-dimensional array can be generally e5q)ressed as: 

Position of sensor at column c = c • Pcolsw + 

F.(n) 

as well as the sensor-to-sensor spacing within a colmnn (e.g. the row-to-row 
spacing in the y-axis): 

Position of sensor at row r = r • Prow.std + Fy(n) 
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where Pcoi_std is the standard or uniform spacing between columns of sensors, 
Prow.std is the standard or uniform spacing between columns of sensors, n is the number 
of the pixel m the array (e.g. n - (c - 1) + r), and Fx and Fy are functions which 
provide x-axis and y-axis position offset values in a non-uniform manner. 
[00331 For example, Fx and Fy may be a short, predictable pseudo-random number 
patterns or sequences, nonlinear mathematical functions which have broad spectral or 
harmonic content (e.g. ramp or sawtooth functions), or predictable processes such as 
cycHc redundancy check ("CRC") polynomials. 

[0034] Accordmg to the preferred embodiment, a CRC process is used to generate 
the pseudo-random position offset values, as this avoids the need for large look up 
tables and only requires three parameters to "seed" the determination of all the offset 
values. While the intermediate results of a CRC process produce a sequence of 
numbers which are pseudo-random in nature (e.g. non-uniform), CRC processes are 
entirely predictable in that when they are performed over the same set of data, they 
yield exactly the same set of intermediate values. 

[0035] Therefore, a predictable and repeatable number generator can be 
implemented by seeding a CRC process with an initial CRC value, and then 
recirculating the data through the algorithm for n iterations, each iteration of the 
process generating an intermediate value which can be converted into an offset value 
for sensor position n. Then, to recreate the sequence of offset values for a given 
array, the process must only be provided the CRC XOR polynomial, the "seed" value, 

-12- 



AUS920011012US1 Patent AppUcation 

and the conversion method for extracting an offset value from the CRC value at each 
iteration. 

[0036] Table 1 shows the pseudocode for such a CRC process, in which an initial 
CRC value is used to start the process. Then, in each iteration of the offset 
determination, the current value of the CRC is exclusive-OR'ed on a bitwise basis with 
a polynomial, and an offset value is extracted from the modified CRC. 
[00371 This offset value for the n* iteration is the offset value for the n* sensor, 
where n = (c - 1) + r. The offset value is preferrably extracted fcom the CRC value by 
taking certain bits from the CRC value (e.g. he, b4, bs, and bi of an 8-bit CRC), and 
concatenating them to yield an offset value in binary format. Then, this offset value 
can be scaled to range within an acceptable physical offset deviation in either axis. 
[00381 Next, the modified CRC value is barrel shifted left (e.g. shifted to the left 
whereby the most significant bit is moved to the least significant bit) until a "1" is 
obtained in the least significant bit of the CRC. This shifted version of the CRC 
because the CRC value to be XOR'ed in the next iteration for pixel n + 1. 



Table 1: Example CRC Pseudo Random Number Generator Process 



CRC = seed_value 
For pixel = 1 to N 

CRC = CRC XOR polynomial 
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offset = concatentated tap bits of CRC 
Fx(pjxel) = scaled offset 

barrel shift left CRC until least significant bit is a "1" 
next_j)ixel 



[0039] This type of process can be used to generate the x-axis offsets as well as the 
y-axis offsets. Preferably, different seed values are used for x-axis offsets and y-axis 
offsets to provide a more random and non-uniform distribution of the sensors. 
[0040] For example, consider a two-dimensional CCD array which is 10.46 mm 
wide havmg 754 sensor columns, and 3.38 mm tall with 244 sensor rows, 
with each sensor being 6.0 \im square (e.g. one-half of a sensor array for a standard 
780 by 488 sensor array). In this case, a traditional uniformly spaced two-dimensional 
array would have the columns of sensors spaced uniformly at approximately 14 fun 
apart, and the rows are uniformly spaced at the same 14 nm. For the purposes of this 
example, also assume that the minimum spacing between adjacent sensors is 4 jun 2.0 
^im to maintain a desired level of anti-blooming isolation. For non-square sensors and 
arrays in which row-to-row spacing and column-to-column spacing are not equivalent, 
different values of these geometries may be accommodated. 

[0041] To reaUze a two-dimensional array of similar resolution (e.g. same number of 
pixels where N = 754 * 244 = 183,976), a pseudo-random non-uniform number 
function (PSNUF) is defined using an 8-bit wide CRC (b7b6b5b4b3b2bibo with b, being 
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the most significant bit) seed value of b'l 1010010 (h'D2) for the x-axis offset, a seed 
value of b'lOl 101 1 1 (h'B7) for the y-axis offset, and an XOR polynomial of 
b'lOOl 101 1 (h'9B), where b' denotes bmary notation and h' denote hexadecimal 
notation. 

[0042] Further, the offset values in this practical example may be determined by 
"tapping" and concatenating b6b4b3bi from the CRC value in the iteration for the 
sensor n to produce a 4-bit offset value ranging non-uniformly between and including 
0 and 15 in decimal notation. In Ms example, we may use offset values 0 through 7 
to represent left and down offsets from the standard grid, and values 8 through 15 to 
represent right and up offsets from the standard grid. Alternatively, another bit from 
the CRC could be tapped as a sign bit, with left/down offsets being negative and 
right/up being positive, etc. 

[0043] Finally, the offset values are scaled to fit the maximum allowable offset 
range. In this example, if the sensor fabrication process requires a minimum spacing of 
2.0 pm to maintain a desired level of anti-blooming isolation, the variance from 
uniform center spacing which is allowable in tiiis situation is 14 \im less 6 \mi less 2 
pm, or 6 ^m. So, the scaling process of Table 2 can be employed to determine the 
physical offset of a particular sensor. 
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Table 2: Example Offset Scaling Process 



If 0 < offset < 7 then 

direction = left or down 
offset^distance = (offset / 8 ) * Poffset jnax 

else 

direction = right or up 
offset^distance = [(offset - 7) / 8 ] * 

endif 



[0044] So, in this example, the first few x-axis offset values given the x-axis seed 
15 value, polynomial and tap bits, would be 10, 7, 9, 13, 9, 7, 15, etc., for the first, 

second, third, etc., sensors. After scaling, the physical x-axis offsets would be right 
2,25 ^m, left 5.25 ^m, right 1.5 jiim, right 4.5 urn, etc., for the first, second, third, 
etc., sensors. 

[0045] For the y-axis offset values, the first few offset values (using the y-axis seed 
20 value) would be 2, 15, 1 1, 1, 3, 11, 4, 7, etc., for the first, second, third, etc., sensors, 
which scales to down 1.5 urn, up 6 |Ltm, up 3 pm, down 0.75 |jm, down 2.25 jmi, etc. 



10 
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[0046] Employing such a distribution function, a new two-dimensional array is 
preferrably fabricated with tiie non-umformly spaced sensors according to fee 
non-uniform functions and Fy, instead of the uniform spacing of prior art sensor 
arrays. This is preferrably done in the silicon design and layout, such as the phase of 
design of a CCD sensor. This allows for the manufactured sensor to incorporate the 
sensors with the x-axis spacing of the function F, and y-axis spacmg of the function Fy, 
such that when it is used for imaging, moire patterns will not form in the image. 
[0047] After the image sampling is complete (or concurrently with data sample 
collection), a linear interpolation may be performed between each non-uniformly 
spaced data sample to create synthesized data values for a two-dimensional 
uniformly-spaced data set. For example, as shown in Figure 7, for a pixel at column n 
in a given row in a uniformly spaced data set (68), the closest surrounding data values 
from the non-uniformly spaced data set (64) may be used to calculate its value using 
linear interpolation as such: 

E„ = {(dieft / dieft + dright) • (Fright - Eielt)} + Eieft 

where E„ is the data value (energy, brightness, etc.) for Hie interpolated pixel 
on the given row, d,eft is the distance from the interpolated pixel to the closest actual 
data sample to the left (in the x-axis) and Eua is the data sample value of that left 
neighbor; drfght is the distance from the interpolated pixel to the closest actual data 
sample to the right (in the x-axis) and Enght is the data sample value of that right 
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neighbor. This produces a linearly interpolated value along the x-axis only, which is 
sufficient for many applications. In some applications, however, a nonlinear 
interpolation may be in order (e.g. when using sensors with a nonlinear sampling 
characteristic), and interpolation considering adjacent row data samples in the y-axis 

5 (e.g. above and below) may be employed. 

[0048] So, an array of evenly spaced data samples (68) for c = 1 to C columns and 
for r = 1 to R rows, is preferably generated from the dithered data set (64), which can 
then be readily processed by common image compression and decompression 
technologies such as JPEG and MPEG utilities. 

10 [0049] By employing the spacing technique of the sensors in the two-dimensional 
sensor array, a two-dimensional image data set which is free of inherent sensor spacing 
frequencies can be obtained which avoids the creation of moire patterns in the first 
place. 

[0050] According to an advanced realization of the invention, each non-uniformly 
15 spaced sensor array is assigned a reference identifier which is associated with a 

particular distribution ftmction and seed values. In this way, a sensor manufacturer 
may produce a number of sensors having the same distribution pattern, and users of 
those arrays may interpret the data according to the manufacturer's specification of the 
distribution function. For the two-dimensional array of the previous example, the seed 
20 values could be combined with the polynomial value(s) to identify its distribution 
scheme, such as .D29B-B79B. 
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[0051] It is fiirther recommended that data files containing samples from such arrays 
and sampling processes be identified, either within the file (e.g. a header area) or in 
naming convention, so that other processes may properly identify the distribution 
fimction of the sample data. 
5 [0052] Further, industry consortiums could determine standards for distribution 
functions such that multiple manufacturers could produce "compatible", equivalent or 
interchangeable sensor arrays and data files. 

[0053] While certain details of a preferred embodiment have been disclosed, and 
S certain examples have been given to illustrate the invention in a more understandable 

I 10 manner, it will be recognized by those skilled in the art that the spirit and scope of the 

; Ji present invention is not limited to these examples and embodiment details. For 

« example, certain terms including x-axis, y-axis, row, column, left, right, above and 

f U below, have been adopted to give a frame of reference. These terms, however, should 

not be interpreted in a limiting manner, as other terms such as horizontal/vertical, 
15 radial/ angular, nortli/southy^east/west, etc., can be used to describe the invention in 
other frames of reference. 

[0054] Further, the invention may be employed for a wide range of information 
gathering and imaging applications, not just photographic and visible spectrum 
imaging, such as magnetic resonance imaging, radar and sonar imaging, etc. 
20 Therefore, the scope of the present invention should be determined by the following 
claims. 
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